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PATENT APPLICATION 
IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 



In re application of 

Manfred SPITZER, Stefan HAHN, Markus 
HESS, Ulrich KRESSEL, Siegfried ROTHE 
and Walter ZIEGLER 

Appln. No . : 

Filed: October 17, 2000 

For: SYSTEM FOR ASSISTING THE OPERATOR OF 
A TECHNICAL APPLIANCE 

Attorney Docket No.: 3926.016 



PRELIMINARY AMENDMENT 

Box: PATENT APPLICATION 

Honorable Commissioner of 

Patents and Trademarks 
Washington, D.C. 2 0231 

Sir: 

Prior to examination of the above- identified application, 
please amend the application as follows: 

IN THE SPECIFICATION : 

Page 1, line 2, insert: 

--BACKGROUND OF THE INVENTION 

Field of the Invention--; 

Page 2 # delete the first full paragraph (lines 9-10); 
Page 2, line 11, insert: 

--SUMMARY OF THE INVENTION--; 

Page 6, line 12, insert: 

--BRIEF DESCRIPTION OF THE DRAWINGS--; 

Page 6, line 23, insert: 
--DETAILED DESCRIPTION OF THE INVENTION--. 



U.S. Application No.: 
PRELIMINARY AMENDMENT 

ATTORNEY DOCKET: 3 926.016 

IN THE CLAIMS : 

Page 10 7 line 1, please delete "PATENT CLAIMS" and insert 
therefore --What is Claimed is:-- 

Please cancel claims 1-15 and add the following new claims: 

--16. System for assisting an operator of a technical 

appliance, said system including an image display device for 
displaying information in the form of an image or symbol in the 
normal field of vision of the operator in situations requiring 
reaction of the operator, wherein said image or symbol (1) is 
displayed at least once for a brief period of time, and wherein 
the duration of the display is below the threshold of recognition 
and above a subconscious threshold of detection of the operator. 

17. System according to Claim 16, wherein the operationally 
relevant information is displayed briefly immediately after 
recognition of the appropriate situation. 

18. System according to claim 16, wherein the operationally 
relevant information is warning information. 

19. System according to claim 16, wherein the technical 
appliance is an automotive vehicle. 

20. System according to Claim 16, further including 

at least one device for automatic recognition of situations 
which may require the reaction of the automotive vehicle operator 
and 
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U.S. Application No.: 
PRELIMINARY AMENDMENT 



ATTORNEY DOCKET : 3 926.016 



a device for controlling the image display device (2) to 
display appropriate operationally relevant information when the 
device for automatic recognition recognizes an appropriate 
situation. 

21. System according to Claim 20 , wherein the situation to be 
recognized is a danger situation. 

22. System according to Claim 20, wherein the at least one 
device for automatic recognition of situations is one or more of 
the following devices: 

a) a device for recognition of an inappropriate speed of the 
vehicle prior to a curve, 

b) a device for recognition of traffic signs, 

c) a device for recognition of road conditions, 

d) a device for recognition of traffic signals 

e) a device for recognition of approach situations, 

f) a device for recognition of crossing situations, 

g) a device for recognition of pedestrians, 

h) a device for recognition of low signal amplitude and tedious 
driving situations. 

23. System according to Claim 20, wherein the at least one 
device for automatic recognition of situations includes a 
navigation device and/or a device for monitoring operational 
values of the vehicle. 

24. System according to Claims 20, wherein the briefly displayed 
images or symbols (1) include one or more of the following 
objects : 

a) images of appropriate situations, 
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U.S. Application No.: 
PRELIMINARY AMENDMENT 

ATTORNEY DOCKET: 3 926.016 

b) pictograms, 

c) traffic signals, 

d) arrows, 

e) written words. 

25. System according to claim 16, wherein the image display 
device is a heads-up display. 

26. System according to Claim 25, wherein the heads-up display 
is wearable by the operator in the manner of at least one of 
glasses or goggles. 

27. System according to Claim 16, wherein the image display 
device is a display screen (2) for direct observation by the 
operator . 

28. System according to Claim 27, wherein the display screen is 
a display screen (2) with controllable pixels on the instrument 
panel for representation of operating parameters of the vehicle. 

29. System according to claim 16, wherein the duration with 
which a respective image or symbol (1) is briefly displayed is 
adjustable by the operator. 

30. System for assisting an operator of a technical appliance, 
said system including an image display device for displaying 
operationally relevant information in the form of an image or 
symbol in the normal field of vision of the operator, wherein 
said image or symbol (1) is displayed at least once for a brief 
period of time, and wherein the duration of the display is below 
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3926.016 



the threshold of recognition and above a subconscious threshold 
of detection of the operator. 



The specification and claims have been amended to conform 
the original translated specification and claims to U.S. 
requirements, i.e., appropriate section headers are added, 
reference in the specification to the claims have been amended in 
order to eliminate multiple dependent claims and claims 
improperly depending from multiple dependent claims, and to 
otherwise conform the claims to U.S. practice. Care has been 
taken to ensure that no new matter is added to the text. 

Entry and favorable consideration prior to consideration are 
respectfully requested. 



REMARKS 



PENDORF & CUTLIFF 
P.O. Box 20445 
Tampa, Florida 33622-0445 
(813) 886-6085 




Date: October 18, 2000 
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EXPRESS MAIL CERTIFICATE 



" EXPRESS MAIL" MAILING LABEL NUMBER: EL568148581US 

DATE OF DEPOSIT: October 18, 2000 

I HEREBY CERTIFY that the foregoing PRELIMINARY AMENDMENT is 
being deposited with the United States Postal Service "Express 
Mail Post Office to Addressee" service under 37 CFR 1.10 on the 
date indicated and is addressed: ATTN: BOX PATENT APPLICATION, 
Commissioner of Patents and Trademarks, Washington, D.C. 20231. 

The Commissioner is hereby authorized to charge any 
additional fees which may be required at any time during the 
prosecution of this application without specific authorization, 
or credit any overpayment, to Deposit Account Number 16-0877. 




Sherri A. Campbell 
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PATENT APPLICATION 



IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 

In re application of 

Manfred SPITZER, Stefan HAHN, Markus 
HESS, Ulrich KRESSEL, Siegfried ROTHE 
and Walter ZIEGLER 

Appln. No. : 

Filed: October 17, 2000 

For: SYSTEM FOR ASSISTING THE OPERATOR OF 
A TECHNICAL APPLIANCE 

Attorney Docket No.: 3926.016 

VERIFICATION STATEMENT PURSUANT TO 37 C.F.R. §1*68 

Box: PATENT APPLICATION 

Honorable Commissioner of 
Patents and Trademarks 
Washington, D.C. 20231 

Sir: 

I, Stephan A. Pendorf, declare and state the following: 

I am a citizen of the United States residing at 3 94 0 
Venetian Way, Tampa, Florida; 

I have lived in Germany for 14 years and am familiar with 
both the German and English languages and have experience as a 
technical translator; 

The attached English- language document is a full, true and 
faithful translation made by me of the text of the attached 
German- language document identified with Attorney reference P 
111 869. 

I hereby declare that all statements made herein of my own 
knowledge are true and that all statements made on information 
and belief are believed to be true; and further that these 
statements were made with the knowledge that willful false 
statements and the like so made are punishable by fine or 



Patent Application 
Verification Statement 

Attorney Docket: 3926.016 

imprisonment, or both, under Section 1001 of Title 18 of the 
United States Code, and that such willful false statements may 
jeopardize the validity of this application and of any patent 
issuing thereon. 



Date : 
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DATE OF DEPOSIT: October 18, 2 000 

I HEREBY CERTIFY that the foregoing VERIFICATION STATEMENT 
and a stamped receipt post card are being deposited with the 
United States Postal Service "Express Mail Post Office to 
Addressee" service under 37 C.F.R. §1.10 on the date indicated 
and is addressed: ATTN: Box PATENT APPLICATION, Commissioner of 
Patents and Trademarks, Washington, D.C. 20231. 

The Commissioner is hereby authorized to charge any 
additional fees which may be required at any time during the 
prosecution of this application without specific authorization, 
or credit any overpayment, to Deposit Account Number 16-0877. 

S Sherri A. Catrfcbell 
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SYSTEM FOR ASSISTING THE OPERATOR OF A TECHNICAL APPLIANCE 



The invention concerns a system for assisting an operator of a 
technical appliance, with an image display device, of which the 
5 image is provided within the normal viewing field of the 
operator, for indication of operationally relevant information in 
the form of images or symbols. 

Modern warning systems are being designed to warn the automobile 
*10 operators of dangerous situations, of which the recognition has 
hitherto been the sole responsibility of the driver. For 
example, a "curve warner" provides the operator with early 
warning of the encountering of curves, which are too tight for 
the actual driving speed. Presently, this warning occurs using 
15 acoustic enunciators, for example "sharp left curve ahead," 
and/or by offering optical warning signals in the dashboard or 
reflected into the windshield. 

This type of warning system is, as a rule, so designed, that the 
20 warning of a danger only occurs when it is to be presumed that 
the driver has not himself recognized this danger, since 
otherwise the acceptance and the effect of the warning 
information would be ignored or lost. In many warning systems, 
it is however difficult or even impossible to determine or fix a 
25 suitable time point for generating the warning. On the one hand, 
the warning must occur early enough so that the driver can be 
appraised of the dangerous situation, should he not have 
recognized it himself. On the other hand, the warning should not 
occur so early, that a driver, who has already recognized the 
30 danger and intends to take appropriate measures, is unnecessarily 
disturbed or distracted. These two requirements can frequently 
not be accommodated at the same time. 

The same dilemma occurs in warning or information systems for 
35 other technical apparatus other than automotive vehicles . For 
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example, the operator of a machine or industrial apparatus is 
normally optically or acoustically warned or informed when 
particular situations occur which might possibly necessitate a 
rapid intervention. Too early or, as the case may be, too many 
5 warning signals could lead thereto that the attentiveness of the 
operator is reduced with time and important information is 
disregarded. On the other hand, the warning should not occur so 
late, that the operator might possibly not be able to react. 

-10 The above -described problem is solved by the characteristic 
features set forth in Patent Claim 1 . 

The invention is accomplished with providing the optical 
information for such a short period of time, that it is not 
15 consciously recognized, however is registered by the person. 
This phenomenon, referred to as "visual priming, " is described 
for example by A. J. Marcel in Conscious and unconscious 

perception: Experiments on visual masking and word recognition, 

Cognitive Psychology 15 (1983), Pages 197-237. The best known 
20 example of this phenomenon is the exchange of individual frames 
of a film sequence with frames having a completely different 
content, which the observer does not consciously comprehend 
during watching of the film, but which however do affect his 
perceptions or, as the case may be, his behavior. 

25 

This effect is used in the invention for sub-threshold warning, 
informing, or influencing of the operator of a machine. This 
makes possible for example a warning without distracting signals 
such as the illumination of lights or warning noises with 
30 conventional warning systems. 

During experiments for carrying out the invention, it has been 
demonstrated that the sub-threshold offering of operationally 
relevant information in an image display, of which the image is 
35 in the normal visual field of the operator, effectively 
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sensitizes the operator to the offered information. Thereby, the 
reaction time of the operator is substantially reduced when the 
situation, to which he was unconsciously prepared, actually 
occurs . The operator is however not disturbed or distracted by 
5 the information offered on the sub- threshold level. Thus, the 
operationally relevant information can immediately be displayed 
for a short period of time after the appropriate situation has 
been recognized, that is, without consideration as to whether the 
operator has initiated any manipulations. 

■10 

According to the invention, an image or symbol, which reflects 
the operationally relevant information, is displayed at once for 
a short period of time, wherein the duration of the display is 
preferably below that of conscious recognition and above that of 

15 subconscious recognition of the operator. Therein, the threshold 
of subconscious recognition is defined as the display duration 
with which the information is just barely accepted unconsciously, 
that is, has an effect on the later behavior of the operator, and 
the conscious recognition threshold is the duration of display at 

20 which the operator becomes conscious of the information. 

These two recognition thresholds can, by the way, be influenced 
by the size and magnitude or intensity of the displayed image 
and, besides this, by whether the image or symbol is offered in 
25 the center of the field of vision or at the edge . It has been 
shown that a short time presentation in the periphery of the 
visual field influences the cognition or recognition or, as the 
case may be, the behavior of the operator. 

30 Basis on the condition that the operationally relevant 
information could be offered even in the periphery of the field 
of vision of the operator, the invention is suitable also for 
technical appliances, such as for example, an automobile. 

35 When adapting the invention to an automobile, one employs 
normally one or more devices for automatic recognition of 
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situations, which may possibly require a reaction of the 
automobile driver, and a device for controlling the image display 
device, in order to display the appropriate operationally 
relevant information as soon as the appropriate situation is 
5 recognized. 

For recognition of dangerous situations, there can be employed 
for example devices for recognition of an inappropriate speed of 
the vehicle prior to a curve, for recognition of traffic signs, 
-10 for recognition of road conditions, for recognition of traffic 
signals, for recognition of approach situations, for recognition 
of crossing situations, for recognition of pedestrians and/or for 
recognition of low stimulation or signal intensity and tedious 
driving situations. 

15 

The invention is suitable not only for sensitization with respect 
to dangerous situations, but rather also to sensitize automobile 
drivers or operators of other technical apparatus to various 
other important conditions without imposing or disturbing, i.e. 

20 conditions which might possibly require a reaction. Devices for 
recognition of various possible important situations during the 
operation of a vehicle, which can advantageously be combined or 
utilized with the present invention, are for example a navigation 
device and/or a device for monitoring of operational values of 

25 the vehicle, for example the status of the fuel. 

The recognition signals of the above-mentioned devices can in 
many ways be combined with each other, in order to prepare the 
information and in present it in an optimal manner to the driver. 

30 For example one symbol may be a symbol, which alerts the driver 
below the threshold of recognition to the possible need of a fuel 
stop, and can be given exactly then when the navigation system 
determines that a filling station is being approached. The 
briefly displayed images or symbols can be pictures of 

35 corresponding situations, pictograms, traffic symbols, arrows or 
written words . 
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In a preferred embodiment, the image display device is a so- 
called heads -up display. In an automobile, one realizes a heads - 
up display thereby that the image information is reflected into 
5 the visual field of the driver via the windshield. The reflected 
image information can in a first stage or level be the 
conventional operating parameters such as RPM, speed, blinker, 
etc. and the operationally relevant information according to the 
invention can supplementally be reflected or reflected during an 
10 appropriate short time fade-out of the main image. 

The invention is also suitable for retrofitting to vehicles, 
which have for example conventional individual instruments on the 
instrument panel. In such a case, it can be useful when all the 
15 short time images are reflected via the windshield into the 
visual field of the driver. 

For adapting a motorcycle or other machine or vehicle, which has 
no suitable reflecting device such as a windshield, a heads-up 
20 display can be designed in the manner of, for example, glasses or 
goggles wearable by the operator. 

In place of a heads-up display, the image display device can be a 
pixel based controllable display screen for direct viewing by the 

25 operator, for example an LCD-display on the instrument panel, 
upon which important operational parameters of the vehicle are 
represented. As already described, the briefly offered images, 
even when peripherally displayed, leave behind a measurable 
impression with the driver, so that it is not detrimental when he 

30 normally does not look directly at the image screen or, as the 
case may be, in complex driving situations frequently changes his 
direction of viewing. 

For safety, the short time images can also be offered multiple 
35 times sequentially at spaced-apart time intervals. This is 
possible, since in accordance with the invention it is not 
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necessary to wait to see whether the driver reacts by himself . 
Rather, all necessary warning or other information can be given 
immediately, so that substantially more time latitude exists than 
with conventional warning systems. 

5 

It is useful when the duration, with which the respective image 
or symbol is displayed for a short period of time, can be 
adjusted or controlled by the operator. Thereby, the operator 
can adapt the system precisely to his personal threshold of 
•10 recognition. In order to facilitate this, the system can offer 
an appropriate adjustment procedure, through which the operator 
is guided. 

Further characteristics and advantages of the invention can be 
15 seen from the following description of embodiments and from the 
figures, to which reference is made. There is shown: 

Fig. 1 - a flow diagram for illustrating the process steps in a 
system for assisting the operator of a technical appliance using 
20 sub-threshold display of information, and 

Fig. 2 - a test apparatus for determining the effectiveness of 
the invention. 

25 A system for assisting a vehicle operator includes for example a 
device for recognition of an inappropriate speed of the vehicle 
prior to a curve, a device for recognition of traffic signs, a 
device for recognition of the road conditions, a device for 
recognition of traffic signals, a device for recognition of 

30 approach situations, a device for recognition of crossing 
situations, a device for recognition for pedestrians and/or a 
device for recognition of difficult to recognize and tedious 
driving situations. 

35 Besides this, there is a freely or variably programmable display 
device in the vehicle, for example an LCD-display in the 
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instrument panel, on which the operating parameters such as for 
example, speed, RPM, etc. are displayed. Alternatively, the 
display device can be a heads -up display. 



5 As shown in Fig. 1, upon recognition of a particular environment 
or situation (SI) the recognized situation is evaluated (S2) with 
respect to its danger potential or, as the case may be, its 
importance. Subsequently, there occurs a selection of a suitable 
excitation or stimulation based on type and duration (S3) , 

-10 wherein the appropriate images or symbols and the appropriate 
display duration can be stored, for example in a storage chip or 
component. Subsequently, there occurs the stimulus (S4) , in 
which the image or symbol is briefly displayed on the display 
device. These process steps can be carried out or, as the case 

15 may be, controlled by a suitably programmed microprocessor. 

The respective display durations are determined in advance for 
example by testing. The display duration is selected to be so 
short, that the appropriate image or symbol is just below the 
20 threshold level of conscious recognition of the driver. 

Test Device 1 

A test vehicle with a variously controllable display screen on 
the instrument panel in place of the conventional individual 

25 instruments was so modified, that the reaction of the driver to 
traffic signs and traffic lights could be measured. The reaction 
time of a sufficiently large number of test persons was measured 
once with and once without subconscious optical sensitization. 
For sensitization of the test persons, symbols of the traffic 

30 signals or, as the case may be, traffic lights were displayed 
approximately 50 ms on the display screen, before the traffic 
signs or, as the case may be, traffic lights actually appeared in 
the visual field of the test person. With this subconscious 
sensitization, a significant reduction of the reaction time of 50 

35 ms in comparison to the reaction time without sensitization 
resulted. 
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Test Device 2 

A similar test vehicle with an image display screen displaying a 
combination of instruments was allowed to follow a preceding 
5 vehicle traveling at 20-40 km/h. The test person was required to 
operate the brake pedal as soon as the brake light of the lead 
vehicle was illuminated. Coincidentally, in half the cases, the 
image of two illuminated backlights (vehicle stoplights) was 
blended into the combination instrument, before the vehicle brake 
-10 lights actually illuminated. The duration of the blending in of 
the display of the image was selected to be so short , that the 
driver did not consciously recognize the images. 

Fig. 2 shows the instrument panel of the test vehicle from the 
15 view of the driver at precisely the instant, in which the 
symbolic image 1 of two backlights appear in the combination 

instrument 2 outside the tachometer 3 . The preceding or lead 

vehicle is not shown in Fig. 2. In the cases with subconscious 
sensitization, a significant reduction of the reaction time of 
20 150 ms occurred in comparison to the cases without sensitization. 

The described test systems were designed especially for testing 
the effectiveness of the invention and do not represent a 
commercially suitable warning system. The person working in this 

25 art would, however, recognize that the use of many warning 
systems for vehicle drivers, which are presently available, can 
be substantially improved in accordance with the present 
invention. For example, if one associated the curve warner with 
the present invention in such a manner that the driver is 

30 presented with the clearly recognizable optical warning signal 
(for example the traffic symbol for "curve") mirrored in 
windshield or displayed on the instrument panel with a duration 
below the threshold of recognition. As a warning signal, it 
would also be possible to use the image of a typical curve with a 

35 radius or curvature as detected by the curve warner. 
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If one takes into the consideration the fact that 90% of all 
accidents of vehicles could be prevented if the driver had better 
control over the vehicle in the last 50 0 ms prior to the 
5 accident, then the invention makes it possible, to increase the 
attention of the driver in the decisive phases, without however 
unintentionally disturbing or distracting him. 

In comparison to conventional warning systems, the information 
,10 offered below the threshold of recognition can always be offered, 
even independent of whether or not the operator has actually 
recognized a dangerous situation. For example, in situations, in 
which the vehicle is urgently to be brought to a halt, for 
example at a red light or a stop sign, it is useful, to offer the 
15 warning information independent of whether or not the operator 
has actually taken action or not. 
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PATENT CLAIMS 



1. System for assisting an operator of a technical appliance, 
with a image display device, of which the image is in the normal 
field of vision of the operator, for display of operationally 
relevant information in the form of images or symbols, thereby 
characterized, that the respective image or symbol (1) is 
displayed at least once for a short period of time. 

2. System according to Claim 1, thereby characterized, that the 
duration of the display is below the threshold of recognition and 
above a subconscious threshold of detection of the operator. 

3. System according to Claim 1 or 2, thereby characterized, 
that the operationally relevant information is displayed 
immediately after the appropriate situation is recognized, for a 
short period of time. 

4. System according to one of the preceding claims, thereby 
characterized, that the operationally relevant information is 
warning information. 

5. System according to one of the preceding claims, thereby 
characterized, that the technical appliance is an automotive 
vehicle . 

6. System according to Claim 5, characterized by at least one 
device for automatic recognition of situations, which may require 
the reaction of the automotive vehicle operator and a device for 
controlling the image display device (2) , which displays 
appropriate operationally relevant information, when an 
appropriate situation is recognized. 

7. System according to Claim 6, thereby characterized, that the 
situation to be recognized is a danger situation. 
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8. System according to Claim 7, thereby characterized, that the 
at least one device for automatic recognition of these situations 
is one or more of the following devices : 

a) A device for recognition of an inappropriate speed of the 
vehicle prior to a curve, 

b) a device for recognition of traffic signs, 

c) a device for recognition of road conditions, 

d) a device for recognition of traffic signals 

e) a device for recognition of approach situations, 

f) a device for recognition of crossing situations, 

g) a device for recognition of pedestrians, 

h) a device for recognition of low signal amplitude and tedious 
driving situations. 

9. System according to Claim 6, thereby characterized, that the 
at least one device for automatic recognition of situations, 
which may possibly require the reaction of the automotive vehicle 
operator, includes a navigation device and/or a device for 
monitoring operational values of the vehicle. 

10. System according to one of Claims 6-9, thereby 
characterized, that the short time displayed images or symbols 
(1) include one or more of the following objects: 

a) Images of appropriate situations, 

b) Pictograms, 

c) Traffic signals, 

d) Arrows, 

e) Written words. 

11. System according to one of the preceding claims, thereby 
characterized, that the image display device is a heads -up 
display. 



- 11 - 



12. System according to Claim 11, thereby characterized, that 
the heads -up display is wearable by the operator in the manner of 
a glasses or goggles . 

13. System according to one of Claims 1-10, thereby 
characterized, that the image display device is a display screen 
(2) for direct observation by the operator. 

14. System according to Claim 5 and Claim 13, thereby 
characterized, that the display screen is a display screen (2) 
with controllable pixels on the instrument panel for 
representation of operating parameters of the vehicle. 

15. System according to one of the preceding claims, thereby 
characterized, that the duration, with which a respective image 
or symbol (1) is displayed for a short period of time, is 
adjustable by the operator. 
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